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Abstract 

A sensory world modeling system, congruent with a human 
expert's perception, is proposed. The Experiential Knowledge 
Base (E*KB) system can provide a highly intelligible 
communication interface for telemonitoring and telecontrol of a 
real time robotic system operating in space. Paradigmatic 
acquisition of empirical perceptual knowledge, and real time 
experiential pattern recognition and knowledge integration are 
reviewed [1-5]. The cellular architecture and operation of the 
E*KB system are also examined. 

1. Introduction 

Intelligent robotic decision making is a dynamic interplay of 
continually unfolding processes, characterized by approximate 
reasoning, fuzziness and dynamic readjustment. Problem 
definition and determination of goals, criteria and conditions, 
world modeling, planning and learning, are all contingent on 
perpetual gathering, processing and interpreting of information. 
In these activities, knowledge appears as a precious commodity, 
whose access, refinement and use can extend the ability for 
rational reasoning and problem solving. 

Real time robotic decision making, totally automated or 
human-guided, depends critically on the availability of 
comprehensible internal models of the decision making world. 
Effective operation of adaptive robotic systems requires 
comprehensible man-machine communication with respect to 
cognition and command languages. Intelligible perceptual 
descriptions of the sensory world, based on interactive modeling 
of a human-expert's perception of it, would strengthen rational 
decision making. 

The sensory world's meaningful structure and activity is 
evidenced by the underconstrained and often indeterminate order 
embedded in the sensory data. From the incredible richness and 
complexity of order which is filtered through the limited sensory 
bandwidths of the physical transducers, meaningful structural, 
temporal and causal embedded regularities are discerned and 
interpreted by imposing relevant empirical elastic templates of 
characteristic spatio-temporal norms of reference, which have 
been formed in the course of prolonged and active experience. 

The relevancy of the elastic templates used at any given instant 
is determined by the temporally shifting concerns, attention and 
attitudes of the decision maker. Interactively derived empirical 
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elastic templates for given k-norms, k e K, and their subsequent 
application for real time perceptual recognition and internal 
modeling, are the pursuits of Experiential Knowledge Engineering. 
We call experiential the inferential capture of knowledge by 
direct associative ordering of sensory data, based on elastic 
templates which model a human expert's perception of the sensory 
world. We refer to the empirical elastic templates as "formal 
description schema" - models of the k-norm, denoted as "fdsfc" - 
models [1-5]. 

Human perceptual faculties of mapping discerned sensory order 
into meaningful predicates of probability, possibility or belief, 
function in a much more qualitative manner than one might deduce 
from examining most of the vision research literature, which is 
largely based on the computational framework of optical 
deconvolution, suggested by Marr, Barrow and Tenembaum [6,7]. 
Logic-like deductive reasoning about invariable perceptual 
properties, based on previously established semantics of 
cognition, seem to follow better the biological paradigm [8,9]. 
The fdsfc-model is a consistent, robust and computationally 
efficient tolerance model [1-5]. It assesses quantitative 
conformity to the k-norm, which is also expressible with a 
linguistic label (name) of resemblance to a norm. It functions 
procedurally and progressively on complete or partial data. The 
assessment is compatible with that of the human-expert whose 
perception it reflects. 

In this paper we propose a telerobotic monitoring and control 
system, which is based on pattern recognition and internal 
modeling procedures that perform congruently to the perception of 
a human expert. This approach to telemonitoring and telecontrol 
aims in securing a most comprehensible communication interface 
between the robotic system and the human operator. It is based 
on modeling the empirical perceptual knowledge of a human expert 
in a highly interactive empirical knowledge acquisition system. 
The pattern recognition approach used is "paradigmatic" [9], as 
explained in section 2. The cellular architecture and operation 
of a real time recognition and world modeling system, the 
"Experiential Knowledge Base (E*KB) " system [1-5, and references 
therein] , functioning within the framework of a telerobotic space 
application as a natural extension of a decision maker's organic 
abilities, is also examined in section 3. Remarks follow in the 
concluding section. 

2. Paradigmatic Pattern Recognition 

2.1 Paradigmatic acquisition of empirical perceptual knowledge. 

Pattern recognition is, in essence, an act of inductive 
inference. It is either the act of re-cognition, i.e. of "seeing 
somethingl as something2" [11], or the act of cognition, i.e. of 
forming a class by clustering items with some identifiable common 
properties. Both cases are now accepted as instances of 
recognition and are based on performing inductive inference. 
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Inductive inference, is a process of producing a general 
proposition based on a limited number of particular propositions, 
such that these become special instances of the former (a 
somewhat inexact remark, which however gives a general 
perspective of our argument) . Going from particulars to . 
universal, from concrete to abstract, is also the basic inductive 
function of paradigmatic pattern recognition. In a broad sense, 
this aspect of induction, which includes the advancement of 
argument by analogy, is considered to be the secret of the 
creative process in science. 

The execution of induction involves certain creative 
operations, such as the creation of new hypotheses (abduction) or 
the identification of characteristic features (percepts) of an 
object, which are extralogical . They are acknowledged to be 
outside the domain of logical operations, and obviously outside 
the capability of the digital computer as we have come to know 
it. They, also, are value-dependent and much like the processes 
that underline creation in art. 

A recognizable pattern is an object which is assessed to be a 
member of a class. For this reason, a pattern often plays a 
double role of an individual object and of a class. Logicians 
would define the concept associated with a pattern, or its class, 
by its intension or its extension. The intension is the 
collection of predicates which just defines the concept. The 
extension is the collection of individual objects which 
correspond to the concept and make up the class. In other words, 
the extension is the collection of all the objects that satisfy 
the predicates included in the intension. However, in real life 
situations, we "define” classes by class-samples (paradigms 
[10]). Shown a few samples of a "cat", a small child becomes 
capable of deciding whether any other animal is a cat or not. 

The child is not given the intension or extension of the class of 
cats and he seldom makes a mistake, although he may never be 
capable of defining the intension or extension of the class of 
cats. Theoretically speaking this is not a definition of a 
class. Nevertheless, the procedure generates in the human mind 
the capability of distinguishing a member from a nonmember of a 
class [10]. Excluding clustering, this procedure describes what 
we understand as pattern recognition: having been shown a few 
paradigms (and perhaps a few negative paradigms) of a class, one 
becomes capable of telling whether or not a new pattern belongs 
in this class. Together with the ability of clustering, this 
procedure underlines the schemata of human faculties of 
inferential capture of classified knowledge, i.e. of perceptual 
pattern recognition. 

In paradigmatic pattern recognition which is based on 
interactive modeling of human perception, the class-defining 
elastic templates (the fdsj^-models) are derived neither by 
intension nor by extension. They are derived only by paradigms. 
If the computer is ever to become capable of doing this job, it 
has to derive the class-defining features from the paradigms 
presented to it, so that the task of recognition becomes one of 
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intensional sorting and assessment of conformity, which the 
computer is capable of doing. But, the derivation of the class- 
defining properties from paradigms, is not a machine-feasible 
operation without human aid, as it is based on extralogical 
processes. Also, the introduction into the machine of a scale of 
"distance” for the assessment of conformity/resemblance between 
objects, can not be done meaningfully without telling the machine 
our value- judgments (often including aesthetic, moral, etc, 
judgements) , which are of an entirely extralogical nature. 

Since the inductive faculty can not be formulated as a 
necessary logical law, its implementation in a computer is often 
done by formulating it (by human intervention again) as some kind 
of heuristic principle, like the principle of minimum entropy. 

In the case of paradigmatic recognition, based on modeling the 
perception of some specific human expert, where "attention" may 
be varied at will, we have sought the aid of a human "expert" to 
identify and valuate the percepts of a reference norm, and to 
provide measures and methods for the assessment of 
conformity/resemblance. The interactive procedure for the 
derivation of the fds^-models, and their application in pattern 
recognition, are briefly reviewed below. For more information, 
the reader is referred to [1-5], or directly to the author. 

Samples of norm-patterns and of patterns with perceivable 
deviations from full conformity to the norm are inputted 
externally (as they are available) , or they are generated by the 
modeling system. The human expert is called to verify discerned 
percepts and to assess conformity to the norm of the discerned 
perceptual organizations. The procedure is highly interactive 
and gradually leads to embedding the human expert's perceptual 
knowledge in a fds^-model. The fds-modeling procedure is 
composed of three-phases. Early in the interactive session, in 
conjunction with noise treatment and abstraction operations, the 
"structural identity" of the k-norm, SkID, is established, to 
serve as an object locator and as an early perception device. 

The determination of the norm's percepts and evaluation of the 
elastic parameters for a full -conformity template, the M k -model, 
is done in the second phase. Finally, the determination of 
conformity assessment procedures and measures and the derivation 
of the complete fdsfc-model, are accomplished in the third phase. 
The fdsj^ -modeling procedure is outlined in figure 1. 

2.2 Real time experiential recognition. 

The real time application of the fds^-model by programming a 
computer, in situations where data is sampled sequentially, is 
illustrated in figure 2. The sensory data is sampled and 
inputted serially, value-at-a-time: (Attrp(Objq) ;valuey; 

§x,t;X,T) . In this case the recognition system accumulates an 
advancing window of sensory data, and it repeatedly tests for: 

(a) conformity to boundary or initial conditions, or (b) 
Conformity to structural identity (SkID) , before the 
abstraction-sensitive perceptual overlay (Pk) assessment. In 
case (a) a window of a few successive samples allows advancing 
tests for boundary or initial conditions. Note that although 
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such conditions may be computed and tested progressively using 
only two advancing data-samples, more samples are included in the 
advancing test window, in order to test and establish the 
boundary or initial conditions in the presence of noise and 
measuring errors. The size of the sampling window depends on the 
sampling frequency and on the estimated noise and distortion 
levels. In case (b) , the sampling window is stretched enough to 
allow for a delayed SkID test. 

In situations where data may be generated parallely by a 
sensory "retina", a neural pipeline may be used to implement the 
fdsfc-model , as shown in figure 3 . The fdsfc-model includes a 
great deal of parallel operations of data processing and 
parameter assessment. This fact renders this pattern recognition 
model suitable for highly parallel implementation. Its inductive 
operations of order discernment and conformity assessment also 
make it suitable for realization with neural networks. The 
illustrated scheme for neural pipelined implementation, applied 
to ID retinas, is currently being analysed in the Cybernetics 
Research Laboratory. An added objective in this endeavor is to 
generate real time temporal descriptions of scene events with 
high temporal resolution and small delay. 

2.3 Experiential knowledge abstraction and integration. 

The task of pattern recognition is to discern "lawful" order 
embedded in sensory data, and to interpret it by assessing 
conformity to characteristic constraints (percepts) of a norm. 

The temporal or spatial patterns of some measurable attribute of 
an object, Attr(Obj), are classified in accordance to their 
assessed conformity/resemblance to various reference norms, 
selected for their relevance. In this process, the role of 
abstraction is fundamental. We recognize physical objects by 
recognizing the patterns of their attributes (selected for 
relevance) at some level of abstraction, which provides for the 
depth of detail required for the decision making task at hand. 

The procedural and progressive assessment of conformity to 
percepts of various norms by the corresponding fdsj^— models is 
applied hierarchically to the objects, the attention scenes and 
to the activities at progressive levels of abstraction of the 
decision making world. Rule-based abstraction and integration 
procedures are also used in the implementation of fds^-models, 
and for discernment and interpretation of causal relationships 
and other types of higher complexity order. The use of 
rule-based abstraction is merited because of the nature of the 
experiential knowledge, which is ill-structured, modular, often 
ambiguous and declarative. Such knowledge makes it difficult to 
represent it and process it with semantic networks, using labeled 
arcs and nodes. 

Experiential knowledge is accumulated in real time in timed 
push-down buffers (FIFOs) , weighted with "forgetting" 
coefficients. The short memory stacks function within 
"object-cells" in a cellular organization illustrated in figure 
4. Asynchronous abstractions of temporal and spatial knowledge 
is performed in response to requests from the decision maker. 
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Attention worlds defined by the decision maker specify portions 
of the decision making world for which experiential internal 
models are requested. The requests may be about experiential 
knowledge referring to recent past, present and anticipated world 
scenes and activities. Hierarchically functioning processors 
perform a great variety of concurrent and distributed I/O and 
processing tasks, which implement pattern recognition, 
abstraction and integration operations. 

Both vertical and horizontal abstraction is performed in the 
concurrent and distributed cellular architecture shown in figure 
4. Attribute or object descriptions of higher complexity and 
composition are developed in real time along the vertical 
abstraction hierarchies. Selective cross-cell integration of 
experiential knowledge is performed along the horizontal 
abstraction hierarchy. 

3. The Cellular E*KB system as a Cognitive Prosthesis 

A telerobotic decision making environment for the E*KB system, 
including other decision support components, communication and 
distribution network interfaces, is illustrated in figure 5. The 
decision maker performs tasks of diagnosis, planning and 
subcommand dissemination, being supported by Data Bases, by 
consultation Expert Knowledge Bases that contain various kinds of 
expertise, and by the Experiential Knowledge Base which provides 
a comprehensible man-machine interface through the human 
perceptual channel . The E*KB system acts as a cognitive 
prosthesis to the decision maker's organic abilities for 
recognition and internal modeling of the sensory world. The E*KB 
system [1-5] includes the NOESIS interactive subsystem for 
on-line acquisition of empirical knowledge, the OMIOSIS subsystem 
for capturing and labeling experiential knowledge embedded in 
sensory data, and the ICON subsystem for abstraction and 
integration of experiential knowledge in response to attention 
subworld specifications. Other components of experiential 
knowledge processing, such as cause-effect prediction and 
stability processors, are also included in the E*KB system [1-5]. 
Research work on these subjects, relating to the development of 
an integrated E*KB system, is currently conducted in the 
Cybernetics Research Laboratory and Caelum Research Corporation. 

4 . Remarks 

Reference norms are selected during our long experience guided 
by relevance-driven acts. When we find some relevance-based 
resemblance among various objects, which are perceived in 
connection with some decision making task, we apply a common 
label to them and we classify them in a "norm" class, forgoing on 
their many differences. Resemblance, contiguity in time or in 
space and cause-effect relationships are the kinds of perceptual 
"order" that enter the process of norm-class formation. In this 
paper, we have dealt only with the kind of order related to 
resemblance. 
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The commonsensical view of "classes" is that of a collection 
of "similar" objects. We might say that after centuries of 
scholastic detour, philosophers have come back to this 
simple-minded yet robust commonsensical view. From the theory of 
"general concepts" [Hume, Watanabe, 10] , which is based on the 
concept of similarity, we deal here with those aspects of 
similarity which have to do with "similarity of form" and partial 
conformity to form-relating characteristic perceptual 
constraints. We refer to this kind of similarity as 
"resemblance". In humans, it is a product of association formed 
by mental processes (Hume) , which we model with our procedural 
elastic templates we call "formal description schemata". 

Within the limited space available, we have attempted to 
present the conceptual and the architectural make up of the 
Experiential Knowledge Base system, which can serve as a 
cognitive prosthesis and as a comprehensible communication 
interface in a perceptual telerobot ic application. 
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